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ABSTRACT 



Student progress, in terms of achievement gain, in reading, 
language, and spelling was studied for students in these elementary reading 
programs in the Los Angeles Unified School District: Open Court, Collections 
for Young Scholars, High Achievement /Waiver Schools, and Success for All. The 
study included only second graders with at least 1 year of exposure to all of 
these reading programs and 2 consecutive years of standardized achievement 
data, resulting in samples of 48,132 students with Stanford 9 reading data, 
50,611 with language data, and 52,473 with spelling data. Simple gain score 
(posttest-pretest) was used as the measure of student progress. The study 
sample was divided into low, medium, and high performance groups. To control 
for the impact of school characteristics, these students schools were also 
classified into low, medium, and high scorers on the School Characteristics 
Index (SCI) , a school background ranking index. Two analyses were performed, 
one containing retained students and the other excluding retained students. 
Results show that the Open Court and Collections for Young Scholars programs 
had the most significant impact on reading achievement of low performing 
students. All the reading programs had a significant impact on low performing 
students reading achievement even after controlling for retained students. 
None of the programs had an impact on medium and high performing students 
reading outcomes, and these students actually lost ground. Gains in language 
for students from schools with medium or high SCI ranks were positive for all 
reading programs. (SLD) 
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A COMPARATIVE STUDY OF SECOND GRADE STUDENTS READING, 
LANGUAGE, AND SPELLING GAIN SCORES FOR THE LOS ANGELES 
UNIFIED SCHOOL DISTRICT READING PROGRAMS 



Executive Summary 



Purpose 

The main objective of this report is to examine student progress (gain) in reading, 
language, and spelling for the District’s elementary reading programs. 

Sample 

Since most of the reading programs arc relatively new at least in some schools, this study 
included only second grade students, the only grade in LAUSD with at least one year of 
exposure to all of these reading programs and two consecutive years of standardized 
achievement data. Simple gain score (posttest-pretest) was used as the measure of 
student progress. This required selecting only students with two years or data on 
Stanford 9 reading (N=48,132), language (N=50,61 1), and spelling (N=52,473) 
achievement tests. 

Methods 

Gain scores generally have a negative correlation with pretest data due to regression 
toward the mean and ceiling or floor effects. This means that wc should expect a higher 
gain for initial low achieving students compared to the initial medium or high achieving 
students. One way of controlling for this effect is to divide the selected sample into 
similar initial performance groupings, based on their pretest data. To accomplish this, the 
study sample is divided into three groups: 1) low performing students (student’s pretest 
score is below 1 standard deviation from the mean); 2) medium performing students 
(student’s pretest score is between +/- 1 standard deviation from the mean); and 3) high 
performing students (student’s pretest score is above 1 standard deviation from the pretest 
mean). 

The School Characteristics Index (SCI) is a school background-ranking index created by 
the California Department of Education based on a number of school indicators such as 
percent of English language learners, pupil socioeconomic status, pupil mobility and 
ethnicity, teacher credentialing, class size, and school calendar. To control for the impact 
of school characteristics, the sample schools were classified into three categories: .1) low 
SCI (school rank is below 1 standard deviation of SCI mean); 2) medium SCI (school 
rank is between +/- 1 standard deviation of SCI mean); and 3) high SCI (school rank is 
above 1 standard deviation of SCI mean). 
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Since a significant number of 2nd grade students were retained (n=566l), it was 
important to account for retention. Therefore, the study analyses were performed for two 
conditions: one included retained students and the other excluded retained students. 



Evaluation Questions 

1 , How are 2nd grade students with a similar initial performance level achieving in 
each reading program? 

2. How are 2nd grade students from similar school environments performing in each 
reading program? 



Reading Gain bv Student Initial Performance 

■ Low performing students benefited most from the Open Court (OCR) reading 
program with an average gain of 12.5 and least from Success For All (SFA) with 
an average reading gain of 7.9 NCE units. Excluding retained students, the OCR 
average gain dropped to 7.7 and the average gain for SFA decreased to 5.4 NCE 
points. 

■ Average reading gains for Collections for Young Scholars (CYS) and OCR 
reading programs arc relatively similar for students with initial low performance 
levels. 

■ The average reading gain for low performing students in High Achieving/Waiver 
Schools with school choice programs was smaller than OCR and CYS averages, 
but still substantial. 

■ In all of the reading programs, gain score averages for low performing students 
were positive and meaningful even after excluding retained students. 

* For medium and high performing students (approximately 80% of the sample), 
the average reading gain was cither negative or close to zero (including or 
excluding retained students). 

Language Gain bv Student Initial Performance 

■ When retained students were included, the average language NCE gain for low 
performing students was about the same for all reading programs (about 7 NCE 
points) except for SFA, which was about 3. 1 NCE points. 

■ Excluding retained students from the analysis, the average language gain for low 
performing students becomes either negative or close to zero. This finding 
highlights the importance of considering retention in the analysis. It also 
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indicates that none of the reading programs promotes language achievement for 
non-retained low performing students. 

■ Medium performing students in High Achiever/Waiver Schools with OCR or 
CYS have the highest average language gain followed by High Achiever/Waiver 
schools with school choice programs. The average language gain decreases 
slightly when retained students are excluded. 

• For medium performing students, the average language gain for the SFA program 
was close to zero regardless of the inclusion or exclusion of retained students. 

■ For high performing students, the highest average language gain belongs to SFA 
(3.9 NCE units), followed by High Achicver/Waiver schools using OCR or CYS. 
For this group of students OCR and CYS language gain were the least, although 
positive. 

Spelling Gain by Student Initial Performance 

■ Including retained students in the analyses, the average spelling NCE gains for 
low performing students were about the same for OCR and CYS reading 
programs (about 12 NCE points). The gain for the SFA reading program was 8.1 
NCE points. When retained students were excluded from the analyses, the 
magnitude of spelling gain decreased but remained statistically significant and 
educationally meaningful. 

• For medium performing students, the average spelling NCE gain was higher for 
Open Court compared to other reading programs, including or excluding retained 
students. 

“ For high performing students, the amount of spelling gain was either negative or 
dose to zero, including or excluding retained students. 

Reading Gain bv School Characteristics Index 

• The Open Court reading program was the only reading program with a 
noticeable reading gain for low and medium SCI schools. The average 
reading gain for other reading programs were either negative or close to zero 
especially when retained students were excluded. 

• In high SCI schools, all of the reading programs had a negative or a dose to 
zero average reading gain score with or without retained students. 

Language Gain bv School Characteristics Index 

• In low SCI schools, both OCR and CYS reading programs had positive language 
gain scores. These averages remained positive even after excluding retained 
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students. The average gain for SFA program vanishes when retained students are 
excluded from the analysis. 

• All reading programs had a positive language gain for medium and high SCI 
schools. After excluding retained students, the language gain scores decrease but 
remain positive for all programs. 



Spelling Gain bv School Characteristics Index 

■ Open Court had the highest gain in spelling compared to all other reading 
programs, including or excluding retained students. The average spelling gain was 
positive for all reading programs. 

Summary of Findings 

• Open Court and CYS had the most significant impact on low performing .students’ 
reading achievement. All reading programs have a significant impact on low 
performing students’ reading achievement even after excluding retained students. 

■ None of the reading programs had an impact on medium and high performing 
students’ reading outcomes. Medium and high performing students actually lost 
ground. One interpretation of this finding is that these programs may be too easy 
for these students and not challenging enough to motivate them. On the other 
hand, strong gains in language were evident for the same groups of students in all 
programs except SFA. 

• High performing students benefited most in reading from SFA. One 
explanation for this finding may be that SFA has a variety of reading 
materials more suitable for high performing students. However, it should 
be noted that SFA is utilized in only a few high performing schools. 

• Gains in language for students from schools with medium or high SCI ranks were 
positive for all reading programs. However, the amount of gain decreased after 
excluding retained students. 



Conclusions 



■ Students in the Open Court reading program had the highest positive gain 

especially for low initial performing students. In 25 of the 36 possible analyses, 
the results for Open Court were positive (more than 1 NCE point) and only in 3 
analyses students in Open Court program had a negative gain (less than -1 NCE 
point). 
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■ Students in the Collections for Young Scholars had the next highest positive 
results. In 23 of 36 possible analyses the students’ gain averages were positive, 
and in 9 analyses were negative. 

■ Students in High Achieving/Waiver schools with a school choice loading 
program had a positive gain in 1 8 of the 30 possible analyses. However, in 8 
analyses (27%) the results were negative. 

■ Success for All, and High AchievingAVaiver Schools with Open Court produced 
the least positive results among all reading programs. 
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